00100	SECTION ON WHY ADVANCED AUTOMATION STUDIES
00200		WHY GENERAL PURPOSE ASSEMBLY DEVICES  (TOB)
00300	PROTOTYPE ASSEMBLY STATION  (GENERAL SYSTEM LEVEL--LOU)
00400		MODULAR SUBSYSTEMS
00500		HARDWARE
00600		PERFORMANCE DEMOS
00700		CALIBRATION & SELF-CALIBRATION
00800	RESEARCH IN ADVANCED AUTOMATION
00900	
01000	SYSTEM SUPPORT
01100		HARDWARE
01200		SOFTWARE
     

00100		replace worst jobs first
00200		change the quality of jobs, to human contingency
00300		able to work on a line with humans
00400		avoids necessity for some breakthroughs
00500			not necessary to have radical accuracy improvements
00600			not necessary to revolutionize assembly philosophy
00700	We  assume  that present manipulators have a small but stable market,
00800	and that most will have minicomputers  soon  (Sundstrand  does  now).
00900	Will  the  incremental  cost  of  touch,  force and visual sensors be
01000	offset by the opening of new  capabilities,  and  will  the  cost  be
01100	offset  by  reduced  cost  of  system design for machines which allow
01200	loose alignment tolerances?  We estimate hardware costs in a separate
01300	paragraph,  and estimate that by 1977, the incremental cost of visual
01400	sensors will be roughly $1100, with about $300 additional for a laser
01500	ranging  scheme  as  we  use  now.       With  the  additional memory
01600	requirements, and the multiple of two in  marketing,  visual  sensors
01700	will add $4k to $6k to the selling price, or about 15%-20% of current
01800	manipulator prices.     Presumably labor costs will inflate also,  so
01900	that  cost  comparisons  will  not  change appreciably over that time
02000	scale unless  significant  new  mechanical  designs  are  made.     A
02100	conclusion is that sensor-equipped robots will be available at slight
02200	additional cost.
02300	
02400	We do not expect mechanical production costs to drop dramatically  as
02500	electronics  costs  fall  with integration.   The most promising cost
02600	reduction development would be high-production standard modules  such
02700	as  robots.    As  an example, consider the Stanford arm; produced in
02800	unit  quantity,  parts,  manufacturing   and   assembly   costs   are
02900	approximately  $10k.  The arm is much simpler than an automobile; its
03000	potential  selling  price  in  large   quantity   is   $3k,   without
03100	minicomputer   control.     Potentially,   a  robot  with  two  arms,
03200	minicomputer control, and touch sensors is possible for under $10k in
03300	quantity.  This  is  the  range of price of present transfer machines
03400	which have limited  market  because  they  have  limited  motion,  no
03500	control,  and are thus locked out of the economy of scale.  Versatile
03600	assembly devices have an advantage in utilization in medium scale and
03700	job  shop  production;  their  utilization  efficiency  can be higher
03800	because versatility allows a variety of operations  to  be  scheduled
03900	simultaneously.    The   availability  of  computers  for  production
04000	scheduling will allow the flexibility to be used to  advantage.  With
04100	simple  devices, transfer and assembly operations must be broken down
04200	into small operations  with  comparable  cost  to  a  more  dexterous
04300	system.  Robots can cut down on real estate, since several operations
04400	can be done at one station; belts and other transfer devices  require
04500	more  space than desirable.  A robot which can pick parts from a cart
04600	would save considerable space.  Robots would also  have  longer  life
04700	over  which  to amortize costs, since they could operate with greatly
04800	changed production operations,  and  evolve  with  new  hardware  and
04900	software  modules.   They  would  have higher reliability, since they
05000	would have monitoring, contingency action, and error recovery.   They
05100	would   be   self-aligning,   self-calibrating,   and   self-testing.
05200	There has been difficulty in meeting the tight tolerances necessary
05300	for operation of non-intelligent devices.  It is hard to estimate
05400	the value of systems which work within current industrial tolerance,
05500	rather than requiring radical improvements in engineering practice,
05600	but that continuity does seem important.
05700	
05800	Functional modularity has been emphasized in our studies;  modularity
05900	is  clearly  possible  and  there is a powerful economic incentive to
06000	assemble widely varying systems from one, two, or three arm  modules,
06100	force  and  touch  modules, vision and ranging modules, with planning
06200	and monitoring modules,  computer-aided  mechanical  design  and  job
06300	specification  for robots on a local or network time-sharing service,
06400	with software modularity. This approach to advanced automation  saves
06500	much  wasted effort. The economies of scale in production of assembly
06600	modules are matched by economies of intellectual scale:   the  design
06700	process  is  modularized  and  easily communicable; there is a common
06800	discipline which can be widely absorbed; developments of many sources
06900	can  be  integrated.    The  alternative  is  special design for each
07000	operation.  We expect that at some point, design will be cheaper with
07100	standard  robot modules than with special purpose designs, just as it
07200	is now more economical to use mini-computer controllers than  special
07300	purpose  controllers.    To  some  extent, the dehumanized industrial
07400	worker now fills the role of standard  module,  so  that  we  do  not
07500	expect   discontinuous   change.   The  continuity  with  decades  of
07600	experience with design for  human  versatility  in  assembly  can  be
07700	relied  upon;  robots  with  some  versatility,  sensing, contingency
07800	actions, and dexterity will provide a  transition  to  newer  schemes
07900	which  will  evolve  more slowly. Advances which benefit special case
08000	automation and traditional trends in automation for human  production
08100	will  also  aid general purpose devices.  For example, new techniques
08200	in bonding and fastening will simplify many  assembly  tasks  to  the
08300	capability   of   robots,   while   still   too  complex  for  simple
08400	servomechanisms. An inhibiting factor will be  that  large  companies
08500	who  can  automate  first, will develop systems which will meet their
08600	purposes but which will not be available for general  use.     To  an
08700	extent, though, they may drag their feet to a dangerous degree, since
08800	they rely heavily upon developed systems  available  from  suppliers.
08900	That  suggests  a need for governmental support which will aid medium
09000	scale industry to get over the hump. But given sensor-equipped robots
09100	at  little  extra cost, they will have visual ability to sense color,
09200	strong contrasts in brightness,  recognize  known  objects  from  two
09300	dimensional outlines (top view of flat object) or three dimensionally
09400	using depth, to a certain extent they will be able to recognize three
09500	dimensional  objects using monocular recognition.   They will be able
09600	to visually align  mating  parts  in  assembly  operations,  increase
09700	reliability  by self-calibration and flaw and disaster detection, and
09800	permit large tolerances in parts transfer, even permitting  parts  to
09900	be  taken  from  a  bin.       Several of these abilities are not yet
10000	available in the laboratory, but with known conditions  and  objects,
10100	and  some  engineering (color coding) are achievable within that time
10200	span.
10300	
     

00100	To  bring about a transfer of technology from the laboratory to pilot
00200	production, several concurrent phases are necessary:
00300		systematize hardware (cost conscious)
00400			manipulators and interface
00500			sensor study
00600			standard interface and pan/tilt for visual devices
00700			adopt commercial visual hardware
00800		systematize software and interaction
00900			hand/eye system
01000			drawing program
01100			experience with low/cost graphics
01200			computer-aided mechanical design
01300		ease of problem specification
01400			interaction and symbolic input
01500			visual representation
01600			geometric editor
01700			visual input
01800			perceptual strategies
01900		standardize economical computer configuration
02000			alternative is de facto standards at lower level
02100		provide powerful modules to broaden engineering choices
02200			color coding
02300			2d shape recognition 
02400			3d shape recognition for known objects
02500			3d perception using depth information
02600		design inherent reliability.
02700			continuous self-calibration
02800			disaster detection and contingency planning
02900			wide latitude in tolerances
03000			low incremental cost in quality control
03100			continuous monitoring of function and product quality
03200	
     

00100			VISUAL PERFORMANCE FOR APPLICATIONS
00200	
00300	While there will be some applications for unsophisticated devices, it
00400	seems very worthwhile to develop  advanced  assembly  devices.    Our
00500	research  is aimed in that direction.  One part of our effort will be
00600	vision for assembly; an assembly task will be performed using vision,
00700	touch,  and  two  hands.    The vision effort will be integrated with
00800	other abilities, and while no great generality will be stressed,  new
00900	modules will be incorporated into the system, and a set of techniques
01000	will evolve. These will depend heavily on techniques similar to those
01100	of  our  recent studies in visual feedback and hand/eye coordination.
01200	The internal model of the  world  will  be  used  to  predict  visual
01300	appearance  of selected areas of the scene, and a set of verification
01400	procedures will confirm or contradict the predictions.  Many  of  the
01500	operations will be connected with supplying feedback to manipulation,
01600	for alignment and insertion. These visual procedures will have models
01700	for  the  parts  and approximate locations.  A typical example is our
01800	current project, assembly of a water pump.  The major parts are to be
01900	located,  the  screws  are  supplied  as  is  standard,  by a feeder.
02000	Locating the body and cover of the pump require a gross localization,
02100	then  a  fine  localization  of  screw holes.  Touch provides a final
02200	accuracy and independent coordination of the hand.   The  placing  of
02300	alignment pins within screw holes can be monitored visually; although
02400	the previous actions provide adequate accuracy.  Placing of the cover
02500	over  alignment pins can be carried out without vision.  However, our
02600	improved force  sensing  will  be  useful  for  that  operation  when
02700	completed.  The insertion of alignment pins in other screw holes, and
02800	the insertion of screws in holes will be monitored visually. Although
02900	the  assembly  has  been performed up to tightening of the screws, by
03000	the hand alone, this will provide a good test case for development of
03100	vision, which will increase reliability.
03200	
03300	To  be  precise  in what vision software will be implemented, we will
03400	greatly expand our efforts in visual feedback; at present  these  are
03500	adequate  for  polyhedra.    We will generalize these to objects with
03600	with curved edges, and extend the quality of the verification modules
03700	in   straightforward   ways   using  our  edge  detection  and  color
03800	facilities. These prediction techniques involve 2d shape  description
03900	techniques.   Template  matching  techniques  are straightforward and
04000	standard; we expect to make more  sophisticated  description  modules
04100	which  have  part/whole  capability, of use in planning.  Some simple
04200	extensions to 3d descriptions are planned.  We will extend the  power
04300	of  our  region  finding  and color region analysis programs.  Visual
04400	input devices will be interfaced to  the  minicomputer  and  existing
04500	modules converted.
     

00100		Vision research
00200	
00300	Visual  tasks  connected with assembly will initially be performed as
00400	special case  problems.  
00500	As we have pointed out, communication with the machine will be necessary
00600	for job specification and instruction.
00700	In  parallel with progress on vision for assembly, to  make  these  techniques
00800	available,  we  will  pursue  development of visual strategy programs.
00900	We describe separately the work on construction strategies, and limit
01000	this discussion to visual strategy.  The
01100	domain of assembly  is  particularly  attractive  as  a  subject  for
01200	strategy  programs,  since  the  context  is much richer than in more
01300	artificial environments, such as the blocks world.  Although  a  much
01400	more  complex problem in the shapes and structural relations, in some
01500	ways, it is simpler; there are more types of  visual  information  to
01600	use, more contextual constraints,a greater variety of simple verification operations
01700	to perform.   This step is essential if intelligent assembly  devices
01800	are  to  be conveniently programmed for tasks. Within the division of
01900	labor between large and small machines, this facility must reside  in
02000	the  central  time-sharing  facility  which  has  large  memory, file
02100	storage, and problem-solving languages.  The system must have a model
02200	of  the  parts for assembly, and a priority from higher up of the parts
02300	and relations to verify or perceive.
02400	It will be necessary to have a representation of shape  and  models
02500	for  usual  assembly  operations such as insertion and alignment, and
02600	the ability  to make predictions of expected visual scenes and
02700	choose a sequence of
02800	operations,  all  this  to  enable  convenient communication with the
02900	user.  When these facilities exist, the addition of new features  and
03000	operations  can be incorporated smoothly, so that users can call on a
03100	growing library of planning modules.  The result of a planning effort
03200	is  a  cycle  of  machine planning and interaction which results in a
03300	plan  network  for  the  mini  computer  which  controls   individual
03400	machines.
03500	
03600	
03700	We will continue with our work with description,representation 
03702	and recognition
03705	of complex objects using depth data.  We are able to make complete
03710	descriptions of objects such as rings and s-shaped snakelike objects.
03715	The programs can segment dolls, toy horses, etc in intuitively appealing
03720	ways and describe the segments well.
03800	The next efforts will be to
03805	improve and generalize the segmentation process of describing
03810	primitive parts, to link the descriptions of primitives into
03815	joints, then to form object descriptions from the descriptions
03820	of joints.  
03825	The recognition of such objects depends first on internal modelling
03830	of an object, then on matching perceived objects with previously
03835	stored internal models.  Our representation is useful in giving a
03840	symbolic description of objects which is useful for visual memory.
03845	We intend also to apply large parts of our programs for description
03850	and recognition of curved objects using only monocular intensity
03855	information.  This is a much harder problem, but within our paradigm,
03860	shares much of the same code, and relies on the same conceptual basis
03865	of local 3d models.  This might seem overly sophisticated to be relevant
03870	to industrial robotics, since early applications might use the simplest
03875	facilities.  Our representation of shape provides a natural way of
03880	representing shape internally in symbolic form and in communicating with
03885	a user.
03890	There are other programs at the laboratory in computer graphics (OCCULT)
03892	geometric modelling (GEOMED) which we plan to use as a basis for
03895	Computer-Aided Mechanical Design.
03900	
03905	
03910	
03915	We will continue work on color region analysis and on a new system
03920	which combines region analysis with semantics about objects and 
03925	relations.  These programs already exist, but are under extensive
03927	development to integrate more knowledge and to fit into a
03930	complete visual system.
03935	We anticipate that they will function in more complex environments
03940	than have been attempted in the past.
03945	There is a growing effort on outdoor scenes, in connection with
03950	navigation of vehicles.  While this is not directly relevant to
03955	the industrial environment, it does reflect the growing involvement
03960	in complex visual tasks, and is another force for development of
03965	useful visual programs.
03970	
04700	
     

04800	Much of the research continues from past progress.  Successively more
04900	powerful  and  integrated  visual  feedback  facilities   have   been
05000	integrated, providing coordination between hand and eye,and providing
05100	accurate prediction of features  to  limit  search  in  alignment  of
05200	stacks  and  insertion  into  holes. Those facilities and others have
05300	been standardized and interfaced as modules within a hand/eye system;
05400	systems  of  varying  complexity  can be readily assembled from these
05500	modules.     The  system  is  maintained  in  a   calibrated   state;
05600	self-calibration  procedures  have been developed and are developing.
05700	Colored region finders are in operation;  texture  analysis  programs
05800	have  analyzed  a variety of scenes.  A laser ranging device has been
05900	in operation for almost two years, and  internal  models  of  complex
06000	objects begin
06100	There  will  be more work on modules to be used by these higher level
06200	programs.     We  have  already  developed  prototype  color   region
06300	analyzers.  There  are  new  ideas  to  be programmed, features to be
06400	combined and the whole organized to integrate with strategy modules.
     

00100	A plant would normally have a central time-sharing system which would
00200	monitor and schedule operations of machines and subsystems.  That
00300	provides a natural subdivision of abilities between the small computers
00400	which would control one or more machines, and the central system which
00500	would provide file storage and software support for large programs.
00600	Thus, we would find that the central computer would support:
00700		interactive job specification
00800		mechanical computer-aided design (CAMD)
00900		strategies
01000		complex vision for recovery (interrupt basis from minis)
01100	while the small machines would have, in addition to the arm control
01200	and contingency planning described above:
01300		visual feedback prediction and verification 
01400		edge and line finder
01500		corner finder
01600		region finder 
01700		color regions
01800		2-D template matching of shape
01900		access to large machine for contingency
     

00100	Our past work has included many of  these  functions.  We  have  made
00200	color  region  finders  [Binford,  Bajcsy], texture analysis programs
00300	[Bajcsy], representation of complex objects  [Binford],  laser  depth
00400	ranging devices [Binford, Agin], description of complex objects using
00500	depth information  [Agin],  visual  feedback  systems  for  stacking,
00600	picking up and inserting [Gill, Davis], standardized software systems
00700	[hand/eye system Feldman, Sproull, Pingle], performed sensor  studies
00800	[Binford, Earnest], and maintained calibrated systems [Sobel, Gill].
     

     

00100	It might seem at first that our work on the representation of complex
00200	shapes might have little relevance to the transfer of technology to
00300	industry, since earliest applications might use the simplest facilities.
00400	In fact, one of the earliest needs will be for convenient man/machine
00500	communication.  Present non-intelligent robots are programmed by leading
00600	them through a path which is to be repeated ad nauseum.  An incremental
00700	addition will allow contingency trajectories to be executed in case of
00800	small variations from routine.  The engineer must then specify what
00900	normal function is, and how to monitor it, how to recognize recoverable
01000	situations.  To do so, he can specify this symbolically in interaction
01100	with an input program.  The language in which he specifies these
01200	normal/abnormal descriptions  involves the shape of objects for assembly,
01300	the mating of parts, and schematic descriptions of operations for insertion.
01400	The significance of our representation of shape is that it provides a
01500	natural way of speaking about shape, in terms of part/whole with
01600	primitives which resemble many fabrication operations.  It also provides
01700	considerable intuition for the program as to the intent of the operation,
01800	since it has powerful representation of shape, and an elaboration of
01900	form=function heuristics.  One function of a flat surface is to support
02000	objects (a table is a flat surface at a given height); the function of
02100	a room is to be a box, ie to enclose space.
02200	
02300	Another use of the representation of shape in interaction is for output
02400	of the machine's concepts to the user.  Computer display is a vigorous
02500	field now; our representation provides a more efficient representation
02600	for display and could lead to such facilities being available on 
02700	small machines.  For the machine's perception, this allows greater
02800	use of predictive mechanisms based on internal models.
02900	
03000	The representation aids input in that the information can be conveyed
03100	by human interaction or visual input.  An adequate description of an
03200	object could be to place it in front of the system (or in front of a
03300	big master system) and let the results be inferred.  The representation
03400	is useful for monocular vision also.
03500	
03600	The shape representation is relevant also for automating design of
03700	numerically machined parts.  The specification of machining operations
03800	is relatively straightforward in this framework, and sensor-controlled
03900	machines can be monitored in a much more efficient way.  A particular
04000	feature is that the user and the system would speak the same language
04100	of subparts, which could have variable tolerance for critical parts,
04200	instead of all having identical tolerances.  Lower reject rate and
04300	less waste effort would result from scheduling critical operations
04400	efficiently and selective use of high accuracy.